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SUMMARY 


The  purpose  of  this  exploratory  research  program  was  to  demonstrate  the 
feasibility  of  on-line  classifications  of  brain  waves  associated  with 
specific  visual  stimulus  events.  A  number  of  classification  and  discrimina¬ 
tion  techniques  were  tested  and  compared. 

It  was  found  that  by  utilizing  4  electrodes  placed  on  the  scalp  and 
measuring  the  voltage  amplitude  at  no  more  than  six  time  points,  a  classifi¬ 
cation  accuracy  of  over  90%  was  achieved  for  choosing  between  the  four 
stimuli:  blank  field;  full  field  checkerboard;  upper  half  field  checker¬ 
board;  and  lower  half  field  checkerboard. 

Other  types  of  visual  stimuli  were  utilized  and  their  brain  responses 
classified  using  techniques  developed  for  the  earlier  set  of  tests.  Visual 
stimuli  investigated  included:  stimulation  of  the  upper  and  lower  half  visu¬ 
al  fields  and  the  right  and  left  visual  fields  with  a  checkerboard  pattern. 
Stimulation  of  the  four  visual  quadrants  and  the  full  visual  field  with  a 
checkerboard  pattern.  In  addition,  the  effects  of  focusing  or  defocusing  a 
letter  on  the  brain  potentials  were  examined. \  Also,  an  edge  matching  exper¬ 
iment  was  conducted.  During  this  test,  the  supject  was  shown,  side  by  side, 
two  slides  of  a  photographic  image.  One  of  the  slides,  always  shown  on  the 
left,  was  known  to  be  ground-truth  slide.  Once  this  decision  process  ter¬ 
minated,  both  sides  were  flashed,  the  EEG  recorded  at  this  time,  and  the 
subject  asked  whether  these  2  slides  were  the  same  ones  he  had  previously 
observed. 

■  Classification  results  obtained  with  these  experiments  were  always  in 
the  high  80%  category  or  low  90%.  The  results  obtained  were  very  encourag¬ 
ing  and  suggest  that  these  techniques  may  be  suitable  for  practical  applica¬ 
tion. 
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INTRODUCTION 


The  purpose  of  this  exploratory  research  program  was  to  demonstrate  the 
feasibility  of  on-line  classification  of  evoked  cortical  potentials  associ¬ 
ated  with  specific  stimulus  events.  A  number  of  stimuli  were  selected  after 
consultation  with  personnel  at  the  Visual  Display  Systems  Branch  (AMRL)  and 
a  preliminary  pilot  study  conducted  utilizing  the  selected  stimuli. 

A  number  of  classification  and  discrimination  techniques  were  tested 
and  compared.  A  combination  of  data  preprocessing  for  dimensionality  reduc¬ 
tion  and  use  of  a  quadratic  discriminant  function  yielded  relatively  high 
classification  accuracies. 

Following  the  completion  of  the  pilot  study,  a  number  of  other  experi¬ 
mental  tests  were  performed  at  AMRL  and  the  previously  developed  classifica¬ 
tion  techniques  employed  on  the  new  data.  A  significant  further  improvement 
in  performance  was  obtained  by  prefiltering  the  data  with  a  powerful  minimum 
mean  square  error  (MMSE)  filter  designed  on  the  basis  of  a  smooth  signal 
shape  determined  from  the  latency  corrected  averages  of  the  experimental  da¬ 
ta. 

Specific  Reports  on  Technical  Requi rements 
Task  4_.  1_ 

"Conduct  a  literature  review  and  pilot  studies  for  the  appropriate 
selection  of  chromatic  stimuli.  Variables  to  be  addressed  include  the  in¬ 
tensity,  duration,  and  wavelength  of  the  stimuli,  as  well  as  the  intensity 
and  special  composition  of  the  adaptation  field.  Foveal  viewing  shall  be 
employed  and  all  preliminary  testing  will  be  conducted  at  the  contractor's 
laboratory." 

Literature  Search  and  Selection  of  Stimuli 

Averaging  is  the  most  common  signal  processing  technique  used  to 
enhance  the  signal  to  noise  ratio  so  that  the  various  components  of  the 
evoked  potential  can  be  discerned  and  studied.  Unless  preceded  by  the  words 
single  or  individual,  the  terms  evoked  potential  or  evoked  response  refer  to 
the  average  or  summation  of  cortical  activity  recorded  from  scalp  elec¬ 
trodes.  Evoked  potentials  are  related  to  a  number  of  variables  involving 
the  subject  and  the  stimulus. 

Since  much  data  concerning  the  average  evoked  response  to  various 
vareual  stimuli  is  available  in  the  literature,  and  since  this  study  deals 
with  single  responses,  an  assumption  had  to  be  made  before  the  information 
in  the  literature  could  be  used  to  select  the  visual  stimuli  parameters. 
The  assumption  is  that  the  features  present  in  the  average  evoked  response 
which  enable  pattern  detection  and  classification  will  be  present  to  some 
degree  in  the  corresponding  single  evoked  responses.  In  order  to  simplify 
the  initial  classification  scheme  and  to  offset  the  difficulties  that 
differences  between  single  responses  to  the  same  stimulus  will  present,  a 
set  of  visual  stimuli  that  show  large  differences  in  their  average  responses 
will  be  chosen. 


There  is  a  wide  variety  of  visual  stimuli  to  choose  from.  Color  and 
intensity  are  changeable  characteristics  of  light  itself;  all  transient 
stimuli  will  have  changeable  shape,  size  and  duration;  one  can  also  consider 
patterns  of  varying  degrees  of  discernabi lity,  moving  patterns,  and  stimuli 
presented  to  different  areas  of  the  visual  field. 

THE  EFFECT  OF  ACHROMATIC  STIMULI 

The  average  evoked  response  to  flashes  of  white  light  of  varying  inten¬ 
sity  and  duration  presented  to  both  eyes  is  well  known  and  documented  CRe- 
gan,  1972;  MacKay,  1973;  Riggs,  1972;  Perry,  19693.  Basically,  the  ampli¬ 
tude  of  the  early  components  increases  with  increasing  intensity  (up  to  a 
saturation  point)  and  the  latency  of  the  peaks  decreases  with  increasing  in¬ 
tensity.  This  relationship  is  illustrated  in  Figure  1.  Variation  of  both 
latency  and  amplitude  with  stimulus  parameters  is  important  to  this  study 
because  of  ease  of  classification.  As  an  example,  variation  in  only  the  am¬ 
plitude  of  the  average  evoked  response  would  make  classification  difficult 
because  it  does  not  necessarily  mean  that  the  corresponding  single  evoked 
responses  will  likewise  vary  in  amplitude.  This,  for  instance,  can  be  due 
to  a  large  but  narrow  peak  with  a  different  latency  in  each  single  response, 
resulting  in  a  wide  peak  with  a  small  amplitude  in  the  average  response. 
This  can  also  be  due  to  the  background  EEG  activity  or  "noise",  resulting  in 
amplitude  distortion  in  the  single  evoked  responses,  but  true  amplitude 
representation  in  the  average  evoked  response. 

Frequency  of  stimulation  is  restricted  by  the  latency  of  the  particular 
components  in  the  evoked  response  that  the  investigator  wishes  to  study.  It 
is  possible  to  study  early  components  with  a  higher  stimulus  repetition  rate 
than  with  late  components,  but  empirical  evidence  supports  the  assumption 
that  the  tail  of  one  response  overlaps  and  adds  in  to  the  beginning  of  the 
next  response  at  frequencies  above  three  flashes  per  second  CPerry,  19693. 
The  amplitude  of  the  individual  responses  is  also  dependent  on  stimulus  re¬ 
petition  rate,  being  largest  at  stimulus  frequencies  of  between  8  and  10  Hz 
CRegan,  1972;  Perry,  19693. 

Stimulus  duration  is  also  important  because  it  is  possible  to  discern 
both  "on"  and  "off"  responses  to  stepwise  modulated  homogeneous  fields  of 
white  light  CRegan,  1972;  MacKay,  1973;  Perry,  19693.  A  very  brief  homo¬ 
geneous  flash  produces  a  response  similar  to  the  "on"  response  described 
above  CMacKay,  19733. 

THE  EFFECT  OF  MONOCHROMATIC  STIMULI 

When  the  dimension  of  color  is  added  to  the  above  parameters,  the 
results  are  controversial  and  analysis  is  rather  complex  CRegan,  1972; 
Riggs,  1972;  Perry,  1969;  MacKay,  1969;  White,  19693.  Care  must  be  taken 
that  any  evoked  potential  changes  that  occur  are  truly  due  to  color  effects 
and  not  to  changes  in  luminance.  This  must  be  done  by  including  a  suffi¬ 
ciently  wide  range  of  intensities  to  make  sure  that  any  change  in  the  evoked 
potential  is  a  spectral  rather  than  an  intensity  induced  change.  The  prob¬ 
lem  is  complicated  further  by  many  definitions  of  what  the  spectral  changes 
are.  Figure  2  shows  the  evoked  potentials  of  two  color  normal  subjects  to 
16  colors  of  matched  intensities.  Difficulty  in  precisely  describing  these 
apparently  spectral  dependent  changes  has  hampered  work  in  this  area.  Also, 


Figure  1. 


Evoked  responses  recorded  over  a  5  log  unit  luminance  range  for 
two  subjects.  The  arrows  indicate  the  peak  of  the  wave  for 
which  latencies  were  measured. 


(From  Riggs,  L. 
Psychophysical 


A.;  Wooten,  B.  R.,  "Electrical  Measures  and 
Data  on  Human  Vision,”  Handbook  of  Sensory 
.  7,  Part  4,  1972). 
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Figure  2.  Evoked  potentials  for  16  colors  for  two  color  normal  subjects. 

(From  Regan,  D.,  Evoked  Potentials  in  Psychology,  Sensory, 
Physiology,  and  Clinical  Medicine,  London,  Chapman  and  Hall, 
1972.) 
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subjectively  matching  intensities  to  different  colors  is  of  dubious  quality 
since  there  is  no  experimental  evidence  indicating  correlation  between  spec¬ 
tral  sensitivities  measured  subjectively  and  spectral  sensitivities  measured 
by  evoked  potential  features  [Regan,  19723. 

It  has  also  been  reported  that  color  dependent  evoked  potentials  can  be 
elicited  by  abruptly  changing  colors  while  keeping  the  subjective  luminance 
the  same.  The  results  again  are  dubious  because  of  the  subjective  luminance 
matching  and  because  of  color  adaptation  effects.  Color  adaptation  effects 
have  been  studied  by  using  backgrounds  of  different  colors  and  intensities 
[Regan,  1972;  MacKay,  1969,  White,  19693.  Changes  in  amplitude  of  the 
evoked  potential  seem  to  be  related  to  the  difference  in  wavelengths  of  the 
background  and  stimulus  [Riggs,  19723.  White,  et  al,  differentiated  three 
responses  to  red,  green,  and  blue  stimuli  respectively  in  this  manner 
[White,  19693.  Evoked  potential  correlates  with  stimulus  duration  and  fre¬ 
quency  have  also  been  noted  [Regan,  1972;  Riggs,  19723,  as  well  as  changes 
in  the  evoked  potentials  elicited  by  monocular  and  binocular  color  stimula¬ 
tion  [White,  19693. 

Even  though  many  of  the  discrepancies  reported  in  the  literature  can  be 
ascribed  to  differences  in  stimulus  conditions,  the  results  normally  show 
only  small  changes  in  later  components  (150-300  msec)  as  shown  in  Figure  3 
[Riggs,  1972;  MacKay,  19693.  One  possible  source  of  artifacts  is  that  of 
unintentional  patterns  in  the  stimulus  field.  Patterns  seem  to  produce  a 
large  response  which  can  obscure  the  color  effects  [Regan,  1972;  White, 
19693.  In  general,  the  discrepancies  and  complications  involved  with  color 
related  evoked  potentials  and  the  small  and  hard  to  define  changes  in  the 
responses  make  transient  homogeneous  color  changes  alone  unsuitable  for 
stimuli  in  this  study. 

THE  EFFECT  OF  PATTERNED  STIMULI 

Presenting  the  element  of  contrast  to  the  subject  seems  to  produce  much 
more  consistent  and  interesting  results,  especially  if  the  stimulus  contains 
only  a  change  in  contrast  and  not  in  mean  luminance  [MacKay,  19733. 

Figure  4  shows  typical  patterned  stimuli  along  with  their  average 
evoked  responses.  The  responses  to  the  stripes  and  to  the  blank  flash  are 
rather  similar  whereas  the  response  to  the  checkerboard  pattern  is  larger 
than  the  rest  and  has  different  components  in  the  first  200  milli-seconds. 
Although  these  responses  were  produced  to  stimuli  with  changes  in  both  con¬ 
trast  and  mean  luminance,  the  large  difference  in  the  response  to  the  check¬ 
erboard  pattern  shown  here  gave  it  consideration  to  be  used  as  a  stimulus 
for  this  study. 

Much  work  has  been  done  with  checkerboard  pattern  stimuli.  Changes  in 
responses  to  variations  in  check  size,  contrast,  visual  field,  etc.  have 
been  documented  in  the  literature  [Eason,  1970;  Harter,  1968;  Harter,  1970; 
John,  1967;  Moskowitz,  1974;  Siegfried,  1975;  Nash,  1970;  White,  19693.  One 
of  the  most  striking  effects  is  that  of  check  size.  Check  size  seems  to  af¬ 
fect  the  amplitude  of  the  average  evoked  potential  rather  than  the  latency 
of  its  components  [Harter,  1968;  Siegfried,  19753.  Figure  5  shows  the  ef¬ 
fect  of  check  size  on  the  amplitude  of  the  average  evoked  response.  The 
largest  amplitude  is  obtained  when  the  unit  check  subtends  about  10  minutes 
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Figure  3.  Binocular  responses  to  630,  577,  and  520  nm  stimuli  for  four 
subjects.  Each  trace  is  the  mean  of  two  replications.  The  two 
sets  for  subject  JGM  were  obtained  approximately  one  month 
apart.  Subject  DGC  is  affected  with  deuteranopia  while  subject 
RLK  is  affected  with  amblyopia. 

(From  Perry,  N.  W.;  Childers,  D.  G.;  Dawson,  W.  W.,  "Humaa.  Cort¬ 
ical  Correlates  of  Color  with  Monocular,  Binocular  and  Dichoptic 
Vision,"  Vision  Research,  Vol.  9,  1969). 
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Figure  A.  Responses  to  checkerboard  pattern,  stripe  patterns,  and  to  blank 
flash. 

(From  Rietveld,  W.  J.;  Tordoir,  E.  M.;  Hagenouw,  J.  R.  B.; 
Lubbers,  J.  A.;  Spoor,  A.  C.,  "Visual  Evoked  Responses  to  Blank 
and  to  Checkerboard  Patterned  Flashes,"  Acta.  Physiol . 
Pharmacol.  Neerl .,  Vol.  1A,  1967.) 
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Figure  5.  The  effect  of  unit  check  size  on  the  visual  evoked  potential. 

Mean  amplitudes  for  two  subjects  (right  and  left  scale).  Zero 
unit  check  size  is  a  blank  flash  of  one-half  the  space  average 
luminance  for  other  check  sizes. 

(From  Siegfried,  J.  B.,  "The  Effects  of  Checkerboard  Pattern 
Check  Size  on  the  VECP,"  Bulletin  of  the  Psychonomic  Society, 
Vol.  6(3),  1975.) 


of  visual  angle  [Harter,  1968;  Harter,  1970;  Siegfried,  1975;  White,  19693. 
The  data  here  also  involve  presentation  of  the  pattern  by  changes  in  lumi¬ 
nance  level. 

It  has  been  suggested  that  when  the  presentation  of  the  pattern  in¬ 
volves  changes  in  mean  luminance,  two  different  mechanisms  are  responsible 
for  the  visual  evoked  response:  one  for  the  change  in  luminance  and  the  oth¬ 
er  for  the  change  in  contrast  present  in  the  pattern  CSpekreijse,  1973;  Jef¬ 
freys,  1972],  As  a  result,  some  researchers  have  subtracted  the  response  to 
a  change  in  luminance,  or  have  used  more  sophisticated  stimulus  presentation 
schemes  that  can  present  a  pattern  to  the  subject  without  changing  the  mean 
luminance  CMacKay,  1973;  Nash,  1970;  Spekreijse,  1973;  Jeffreys,  1972;  Jef¬ 
freys,  19713. 

The  effect  of  stimulation  of  various  quadrants  and  octants  of  the  visu¬ 
al  field  by  a  pattern  without  changing  mean  luminance  is  shown  in  Figure  6. 
The  columns  of  evoked  potentials  correspond  to  the  active  electrode  posi¬ 
tions  shown  at  the  bottom  of  the  figure.  Looking  down  the  middle  column, 
part  Ca)  shows  small  amplitude  and  latency  changes  between  the  responses  to 
stimuli  in  the  upper  right  octants  and  quadrant.  Similarly  part  (b)  shows 
small  changes  between  the  responses  to  stimuli  in  the  lower  right  octants 
and  quadrant.  The  midline  electrode  does,  however,  show  a  more  noticeable 
difference  between  parts  (a)  and  (b),  that  is,  the  responses  to  the  stimulus 
of  octants  and  quadrants  in  the  upper  and  lower  right  visual  field.  The 
early  components  (less  than  100  milliseconds)  of  the  responses  to  stimuli  in 
the  lower  right  visual  field  show  positive  peaks,  whereas  the  same  com¬ 
ponents  of  the  responses  to  stimuli  in  the  upper  right  visual  field  show  a 
negative  peak.  This  suggests  that  responses  to  half-field  stimuli  should  be 
investigated  for  large  differences. 

Left  or  right  half-field  stimulation  is  supposed  to  produce  a  large 
response  in  the  contralateral  hemisphere  CMacKay,  1973;  Jeffreys,  19723,  but 
there  is  some  controversy  about  this  CMacKay,  1973D.  Jeffreys  and  Axford 
reported  more  consistency  between  subjects  in  the  responses  to  upper  and 
lower  half-field  stimulation  rather  than  left  or  right  half-field  stimula¬ 
tion  CJeffreys,  19723. 

Figure  7  shows  the  responses  of  six  subjects  to  patterned  stimulation 
of  the  upper  and  lower  halves  of  the  visual  field  (dashed  and  continuous 
lines,  respectively).  Polarity  reversals,  especially  in  the  early  peaks, 
appear  to  be  common  differences  between  average  responses  to  stimuli  of  the 
upper  and  lower  visual  fields  CJeffreys,  19723.  Notice  that  although  the 
responses  for  subjects  J.F.  and  C.M.  bear  some  similarities,  they  are  much 
different  from  the  responses  for  subjects  C.J.W.  or  D.A.J.  This  suggests 
that  the  differences  produced  by  patterned  stimulation  of  the  upper  and 
lower  half  fields  may  not  be  consistent  enough  for  different  subjects. 

The  relationship  between  responses  to  full-field  stimulation  and  half¬ 
field  stimulation  for  various  electrode  positions  is  shown  in  Figure  8.  The 
first,  second,  and  fourth  columns  show  responses  to  upper,  lower,  and  full 
visual  field  stimulation  for  the  electrode  positions  shown.  The  third 
column  is  the  sum  of  the  previous  two  columns.  In  the  light  of  the  previ¬ 
ously  shown  polarity  reversals,  the  remarkable  similarity  between  the  third 
and  fourth  columns  suggests  that  the  full  field  response  may  be  flat  where 
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Figure  6.  Visual  evoked  potentials  to  quadrant  and  octant  stimulation. 

(From  Jeffreys,  D.  A.;  Axford,  J.  G.,  "Source  Locations  of 
Pattern-Specific  ComDonents  of  Human  Visual  Evoked  Potentials. 
I.  Component  of  Striate  Cortical  Origin,"  Experimental  Brain 
Research,  Vol.  16,  1972.) 
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Figure  7.  Visual  evoked  potentials  to  pattern  stimulation  of  the  upper  and 
lower  visual  half  fields  for  six  subjects.  The  responses  for 
two  identical  runs  of  100  averages  are  superimposed  in  each 
case.  Field  size  is  7°  and  pattern  duration  is  100  msec. 

(From  MacKay,  D.;  Jeffreys,  D.  A.  "Visually  Evoked  Potentials 
and  Visual  Perception  in  Man,"  Handbook  of  Sensory  Physiology, 
Vol .  7(3),  Part  B,  1973.)  '  -  - 
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Visual  evoked  responses  for  one  subject  (D-oA.  J->  to  patterned 
stimulation  of  (a)  the  upper  half  of  the  6°  stimulus  field,  (b) 
the  lower  half  of  the  stimulus  field,  and  (d)  the  full  stimulus 
field.  The  sum  of  the  waveforms  for  the  upper  and  lower  half 
field  stimuli  is  shown  in  (c>. 

(From  Jeffreys,  0.  A.,  "Cortical  Source  Locations  of  Pattern- 
Related  Visual  Evoked  Potentials  Recorded  from  the  Human  Scalp," 
Nature,  Vol.  229,  12  February  1971.) 
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the  half  field  responses  show  peaks  of  their  respective  polarities.  This  is 
most  evident  in  the  middle  row  of  Figure  8  where  the  half  field  responses 
are  of  approximately  the  same  magnitude.  Figure  8  shows  yet  another  proper¬ 
ty  that  would  aid  in  classification  of  full  and  half  field  patterned 
responses:  the  upper  half  field  response  is  almost  flat  at  the  lower  extreme 
electrode  position  (below  the  inion),  whereas  the  lower  half  field  response 
is  almost  flat  at  the  upper  extreme  electrode  position  (near  C  )  making  the 
sum  of  the  two  (and  the  very  similar  full  field  response)  n8n  flat  every¬ 
where  along  the  electrode  positions  shown. 

Since  recognizing  a  checkerboard  pattern  composed  of  small  checks  re¬ 
quires  high  visual  acuity,  it  is  intuitive  to  assume  that  foveal  vision 
plays  an  important  role  in  the  patterned  evoked  response  [Jeffreys,  1972; 
Rietveld,  19673.  Figure  9  illustrates  the  importance  of  the  foveal  area  to 
the  checkerboard  pattern  response.  The  top  row  in  this  figure  is  the 
response  to  full-field  stimulation.  The  next  three  rows  show  the  responses 
to  the  same  pattern  with  2,2  5',  and  4  4*  of  the  central  visual  area 
covered  by  a  black  disc.  Since  the  foveal  area  plays  such  an  important  role 
in  the  response  to  checkerboard  pattern  stimulation,  it  would  seem  important 
that  the  subject's  gaze  be  fixed  on  a  precise  point  in  the  stimulus  field, 
particularly  for  upper  or  lower  half  field  stimulation  [Jeffreys,  1972; 
John,  19673. 

Some  work  has  been  done  with  both  color  and  pattern,  but  the  results 
again  contain  the  color  controversy  [MacKay,  1973;  White,  19693,  and  it  has 
been  suggested  that  the  two  mechanisms  are  separate  and  produce  the  sum  of 
both  types  of  response  when  color  and  pattern  are  combined  [MacKay,  19733. 
White  and  others  [White,  19693  abandoned  an  investigation  using  both  color 
and  a  checkerboard  pattern  because  the  pattern  response  seemed  to  obscure 
any  color  effects.  Regan  and  Spekreijse  confirmed  these  results  and  used 
them  to  their  advantage  in  work  with  normal  and  color  blind  subjects  [Regan, 
19743. 

THE  EFFECT  OF  MOTION 

Evoked  potentials  have  been  elicited  by  the  deflection  of  a  pattern  on 
the  retina.  As  a  result,  either  movement  of  the  pattern  or  movement  of  the 
eye  (including  saccadic  movement)  can  elicit  evoked  potentials  [MacKay, 
1973;  Clarke,  1972;  Clarke,  1973a3.  Clarke  has  found  that  evoked  potentials 
eLicited  by  motion  (motion  onset,  motion  offset,  or  motion  reversal)  were 
almost  invariant  to  intensity,  direction  of  motion,  and  sharpness  of  boun¬ 
daries  [Clarke,  1973b3. 

Motion  onset  evoked  potentials  are  much  smaller  than  motion  offset  or 
motion  reversal  evoked  potentials.  For  velocities  below  15  /sec,  the  motion 
reversal  potential  is  very  similar  to  the  sum  of  the  motion  onset  and  motion 
offset  potentials  suggesting  that  the  motion  reversal  evoked  potential  is 
produced  by  independent  motion  onset  and  motion  offset  mechanisms  (Figure 
10)  [Clarke,  1973a3. 

The  visual  evoked  potential  elicited  by  motion  reversal,  motion  offset, 
and  stationary  pattern  appearance  seem  to  have  very  similar  components  and 
scalp  distributions,  and  all  three  show  polarity  reversals  between  upper  and 
lower  half-field  stimulation.  Motion  onset  and  stationary  pattern  disap- 
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The  effect  of  foveal  area  on  the  patterned  response. 

(From  Rietveld,  W.  J.;  Tordoir,  E.  M.;  Hagenouw,  J.  R.  B.; 
Lubbers,  J.  A.;  Spoor,  A.  C.,  "Visual  Evoked  Responses  to  Blank 
and  to  Checkerboard  Patterned  Flashes,"  Acta.  Physiol . 
Pharmacol .  Neer  l .,  Vol.  14,  1967.) 


White  and  others  CWhite,  1969]  abandoned  an  investigation  using  both  color 
and  a  checkerboard  pattern  because  the  pattern  response  seemed  to  obscure 
any  color  effects.  Regan  and  Spekreijse  confirmed  these  results  and  used 
them  to  their  advantage  in  work  with  normal  and  color  blind  subjects  CRegan, 
1 9743 . 


THE  EFFECT  OF  MOTION 

Evoked  potentials  have  been  elicited  by  the  deflection  of  a  pattern  on 
the  retina.  As  a  result,  either  movement  of  the  pattern  or  movement  of  the 
eye  (including  saccadic  movement)  can  elicit  evoked  potentials  CMacKay, 
1973;  Clarke,  1972;  Clarke,  1973a].  Clarke  has  found  that  evoked  potentials 
elicited  by  motion  (motion  onset,  motion  offset,  or  motion  reversal)  were 
almost  invariant  to  intensity,  direction  of  motion,  and  sharpness  of  boun¬ 
daries  CClarke,  1973bD. 

Motion  onset  evoked  potentials  are  much  smaller  ti  an  motion  offset  or 
motion  reversal  evoked  potentials.  For  velocities  below  15°/sec,  the  motion 
reversal  potential  is  very  similar  to  the  sum  of  the  motion  onset  and  motion 
offset  potentials  suggesting  that  the  motion  reversal  evoked  potential  is 
produced  by  independent  motion  onset  and  motion  offset  mechanisms  (Figure 
10)  CClarke,  1973a]. 

The  visual  evoked  potential  elicited  by  motion  reversal,  motion  offset, 
and  stationary  pattern  appearance  seem  to  have  very  similar  components  and 
scalp  distributions,  and  all  three  show  polarity  reversals  between  upper  and 
lower  half-field  stimulation.  Motion  onset  and  stationary  pattern  disap¬ 
pearance  responses  also  seemed  to  be  very  similar,  but  did  not  show  polarity 
reversals  between  upper  and  lower  half-field  stimulation  CClarke,  1973b]. 

When  pattern  appearance/disappearance  and  motion  are  combined,  the 
evoked  potentials  are  velocity  dependent  as  shown  in  Figure  11.  The 
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Figure  10.  Visual  evoked  potentials  to  offset,  onset,  and  revgrsal  of  a 
patterned  field.  Velocities  are  10/sec  (J.C.),  8  /sec  CA.J.) 
and  5°/sec  (S.H.  and  B.M.). 

(From  Clarke,  P.  6.  H.,  "Visual  Evoked  Potentials  to  Changes  in 
the  Motion  of  a  Patterned  Field,"  Experimental  Brain  Research, 
Vol .  18,  1973.) 


pearance  responses  also  seemed  to  be  very  similar,  but  did  not  show  polarity 
reversals  between  upper  and  lower  half-field  stimulation  CClarke,  1973b3. 

When  pattern  appearance/disappearance  and  motion  are  combined,  the 
evoked  potentials  are  velocity  dependent  as  shown  in  Figure  11.  The 
responses  to  pattern  appearance  or  disappearance  are  largest  for  stationary 
patterns  and  all  the  components  are  reduced  as  the  velocity  increased 
CClarke,  197211. 

BASIS  FOR  SELECTION 

As  mentioned  earlier,  variations  in  both  amplitude  and  latency  of 
evoked  response  components  are  important  to  this  study  because  of  ease  of 
detection  and  classification.  However,  some  of  the  largest  changes  in  aver¬ 
age  evoked  potentials  seemed  to  be  the  polarity  reversals  in  certain  peaks 
elicited  by  changes  in  the  stimulus.  The  polarity  reversal  in  the  100  msec 
component  of  the  responses  to  a  white  blank  field  and  to  a  checkerboard  pat¬ 
tern  (Figure  4)  makes  the  responses  easily  distinguishable.  Likewise,  the 
polarity  reversals  shown  in  Figure  7  are  targe  enough  to  make  the  response 
distinguishable. 

As  mentioned  previously,  the  use  of  color  was  ruled  out  because  of  the 
complicated  stimulus  conditions  necessary,  the  controversial  nature  of  the 
results,  and  small  and  hard  to  define  changes  in  the  responses.  The  use  of 
motion  was  also  ruled  out  mainly  because  of  the  great  similarity  between 
responses  to  moving  and  to  stationary  objects.  Also,  it  seems  more  likely 
that  motion  would  tend  to  elicit  more  eye  movement  artifacts,  as  the  eyes 
tend  to  follow  a  moving  object. 

The  response  to  the  appearance  of  a  checkerboard  pattern  without  change 
in  mean  luminance  was  chosen  because  cortical  cells  are  known  to  respond  to 
straight  edges  in  the  visual  field,  and  because  of  the  large  response  it  el¬ 
icits  which  is  easy  to  distinguish  from  the  response  to  the  appearance  of  a 
white  blank  field.  It  has  also  been  suggested  that  these  two  types  of 
stimuli  represent  two  independent  mechanisms  for  encoding  visual  informa¬ 
tion.  The  large  amplitudes  of  the  responses  are  also  advantageous  to  this 
study  because  they  are  indicative  of  either  large  signal  to  noise  ratios  or 
consistency  of  single  evoked  responses  to  the  same  stimulus,  which  are  both 
desirable  effects. 

The  responses  to  the  appearance  of  a  checkerboard  pattern  in  the  upper 
and  the  lower  visual  half  fields  without  a  change  in  mean  luminance  in  these 
fields  also  show  polarity  reversals.  These  two  stimuli  were  also  chosen  be¬ 
cause  they  represent  two  different  locations  in  the  primary  visual  cortex 
and  because  of  the  large  differences  in  their  average  evoked  responses. 
These  responses  will  also  lend  themselves  to  detection  based  on  information 
from  multiple  electrodes  because  different  areas  of  the  cortex  are  involved 
and  because  of  the  difference  in  the  responses  depending  on  electrode  posi¬ 
tion  as  shown  in  Figure  7. 

Task  4^.2^ 

"Following  the  selection  of  the  stimulus  events,  the  preferred  elec¬ 
trode  array  will  be  specified  and  ensemble  average  templates  will  be  gen- 
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Figure  11.  Average  visual  evoked  potentials  (N=100)  to  stationary  (continu¬ 
ous  lines)  and  moving  (dashed  lines)  patterns.  Velocity  is 
8  /sec. 

(From  Clarke,  P.  6.  H.,  "Visual  Evoked  Potentials  to  Sudden  Re¬ 
versal  of  the  Motion  of  a  Pattern,"  Brain  Research,  Vol.  36, 
1972.) 


erated.  Initially,  the  following  configuration  will  be  utilized:  0^,  0^,  0 
(according  to  the  10-20  System)  and  two  other  electrodes  placed  on  midune  I 
cms  above  and  below  0  .  The  ensemble  averaged  data  will  be  used  to  develop 
minimum  mean  square  error  filters.  The  outputs  of  these  processors  should 
characterize  the  individual  evoked  potentials  in  terms  of  component  ampli¬ 
tudes  and  latencies,  as  well  as  overall  morphology.  Several  iterative  pro¬ 
cedures  should  be  instituted  to  derive  better  estimates  of  the  true  signals 
and  hence  more  accurate  filters." 

EXPERIMENTAL  CONDITIONS 

The  subjects  sat  in  a  dark  and  quiet  room  binoc^larly  viewing  a  cathode 
ray  tube  screen  from  45  inches  which  subtended  14  38*  by  12  25 *  of  visual 
angle.  The  subject  viewed  128  stimuli  from  each  of  four  stimulus  condi¬ 
tions: 

(1)  A  step  change  in  luminance  of  the  blank  screen  from  0  to  15  foot 
Lamberts. 

(2)  An  abrupt  presentation  of  a  full  screen  checkerboard  pattern  of 
checksize  10'  and  contrast  24/8  from  a  blank  field.  Space  average 
luminance  remained  constant  throughout  at  15  foot  Lamberts. 

(3)  The  same  as  (2)  above,  but  with  a  black  opaque  piece  of  cardboard 
covering  to  lower  half  of  the  screen. 

(4)  The  same  as  (3)  above,  but  with  a  black  opaque  piece  of  cardboard 
covering  the  upper  half  of  the  screen. 

A  small  cross  in  the  center  of  the  screen  provided  a  steady  central  fixation 
point.  The  stimulus  duration  was  766  milliseconds  and  the  interstimulus  in¬ 
terval  was  3  seconds. 

The  electrodes  used  were  Beckman  miniature  biopotential 
si Iver/si Iverchloride  electrodes.  Four  simultaneous  channels  were  recorded 
from  the  monopolar  electrodes  at  C  ,  P  ,  0  ,  and  the  inion  (International 
10-20  electrode  configuration),  fhe  righ?  earlobe  was  used  as  a  reference, 
and  the  left  earlobe  was  grounded.  Electrodes  were  also  placed  on  the  brow 
and  cheek  so  that  eye  blinks  and  eye  movement  artifacts  could  be  detected. 
A  100  mV  amplitude  10  Hz  sine  wave  was  fed  to  all  four  channels  as  a  cali¬ 
bration  both  before  and  after  the  data  taking  session.  The  data  from  the 
four  channels  were  then  amplified  by  Grass  7P511  amplifiers  and  recorded  on 
a  Honeywell  FM  tape  recorder.  Two  of  the  five  channels  (from  the  Cz  and  Oz 
electrodes)  were  low  pass  filtered  and  averaged  online.  The  data  were  then 
sampled  at  250  samples/second  and  stored  digitally  on  magnetic  tape  in  1 
second  epochs  starting  1/2  second  prior  to  the  stimulus  and  continuing  1/2 
second  after  the  stimulus.  For  the  preliminary  work  it  was  decided  that  two 
subjects  (numbers  409  and  410)  be  tested  according  to  the  procedure  outlined 
above. 


PRELIMINARY  RESULTS 

From  the  online  filtering  and  averaging  mentioned  earlier,  it  was  found 
that  the  data  from  subjects  409  and  410  contained  a  rather  large  amount  of 
60  Hz  noise  from  sources  which  could  not  be  eliminated  at  the  time  the  data 
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were  taken.  Therefore,  it  was  decided  to  filter  the  data  before  doing  any 
other  processing. 

The  data,  originally  sampled  at  250  samples/sec,  were  digitally  fil¬ 
tered  with  a  low  pass  filter  whose  transfer  function  was  zero  for  frequen¬ 
cies  above  60  Hz.  The  frequency  spectrum  of  the  digital  low  pass  filter  ap¬ 
pears  in  Figure  12.  The  phase  characteristic  of  the  filter  is  zero  for  all 
frequencies  concerned  so  that  it  could  not  alter  the  phase  information  in 
the  data  that  might  prove  valuable  to  this  study.  Each  single  response  from 
the  stimulus  onset  to  1/2  second  after  the  stimulus  onset  was  then 
represented  by  125  samples  of  data. 

After  filtering,  the  averages  of  the  first  100  responses  for  both  sub¬ 
jects  were  computed  for  each  electrode  position  and  stimulus  condition. 
These  averages  are  shown  in  Figures  13  and  14.  As  can  be  seen,  there  are 
polarity  reversals  of  the  early  components  for  the  upper  and  lower  field 
checkerboard  stimulus  conditions.  Also,  the  checkerboard  top  half  response 
is  almost  flat  at  the  inion  electrode  and  the  checkerboard  bottom  half 
response  is  almost  flat  at  C :  c.f.  Figure  8. 

CLASSIFICATION  RESULTS 
a)  Correlation  Classifier 

One  of  the  more  basic  and  easy  to  implement  classifiers  is  the  correla¬ 
tion  classifier  or  matched  filter  CFukunaga,  19723.  Consider  M  signals 
s.j  (t) ,. .  .,sM(t)  with  a  priori  occurrence  probabilities  P^,...,PM.  Assume 
that  each  signal  lasts  for  T  seconds  and  that  the  observed  signal  v(t)  = 
s(t)  +  nCt)  is  contaminated  by  additive  white  noise  with  one-sided  spectral 
density  Nq.  The  problem  is  to  decide  which  of  the  M  signals  was  transmit¬ 
ted,  that  is,  to  classify  v(t)  into  one  of  M  classes.  The  correlation  clas¬ 
sifier  computes  M  correlations  and  compares  them  after  adding  M  biases: 

T 

fv(t)  s.(t)  dt  +  b.  (j  =  1,...,M) 

J0  3  1 

where  b^  is  a  constant  bias  given  by 
N  1  T  ? 

b.  =  In  P  -  j  J  s^(t)  dt  (j  =  1,...,M) 


The  correlation  classifier  then  assigns  vCt)  to  the  class  whose  corre¬ 
lation  plus  bias  is  largest.  The  correlation  can  be  implemented  as  the  con¬ 
volution  of  impulse  response  h.(t)  =  s.(T-t>  with  v(t>  where  the  impulse 
response  hj(t)  is  matched  to  tAe  signal  s^.  (t)  hence  the  name  matched  filter. 

For  discrete  time  signals  the  signal  to  be  classified,  X,  is  a  vector 
having  N  components  x^,...,x^  corresponding  to  N  samples  of  v(t),  0  <  t  £  T 
above.  Similarly,  consider  S.  to  be  the  discrete  vector  of  s.Ct)  above. 
Then  the  cross-correlation  Jof  v(t)  and  s.(t)  above  can  be  written  as  $!x, 
and  the  correlation  plus  bias  can  be  written  as  J 
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Figure  13.  Average  Evoked  Responses  for  Subject  409,  N=100. 
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In  implementing  this  classifier,  M  =  4  and  the  £  prioriN  probabilities 
P.j,...,P^  were  assumed  to  all  equal  0.25.  This  makes  the  -=2.  [n  p.  term  a 
constant  for  all  classes  and  it  drops  out  of  the  equation  when  the  Correla¬ 
tion  plus  bias  terms  are  compared.  The  S.'s  are  the  signal  templates  or  ap¬ 
proximations  to  the  signals  so  S.j,...,S^  iere  initially  chosen  as  the  digi¬ 
tized  average  evoked  responses  corresponding  to  a  certain  electrode  position 
and  the  four  stimulus  conditions. 

The  average  response  for  each  stimulus  condition  and  each  electrode  po¬ 
sition  was  then  examined  for  both  subjects,  and  it  was  decided  to  eliminate 
the  last  240  msec  of  data  (keeping  the  first  260  msec)  because  it  conveyed 
very  little  information  to  the  correlation  classifier.  Each  single  response 
was  then  represented  by  65  data  samples.  This  number  was  still  too  large  to 
make  any  but  a  linear  classifier  practical,  so  it  was  decided  to  reduce  the 
data  further  without  destroying  features  necessary  for  classification. 

For  each  single  response,  the  autocorrelation  function  was  computed  and 
these  autocorrelations  were  averaged  to  obtain  an  average  autocorrelation 
functions.  The  Fourier  transform  of  this  average  autocorrelation  function 
was  computed  and  examined  for  each  stimulus  condition  and  each  electrode  lo¬ 
cation  for  both  subjects.  It  was  found  that  these  average  frequency  spectra 
contained  very  little  energy  at  frequencies  above  25  Hz.  The  data  were  then 
digitally  filtered  with  a  low  pass  filter  that  admitted  no  energy  above  25 
Hz.  Since  the  highest  frequency  present  in  the  filtered  data  was  below  25 
Hz,  every  5th  sample  was  selected  from  the  filtered  data  so  that  the  sam¬ 
pling  rate  was  artificially  lowered  from  250  to  50  samples/sec.  Each  single 
response  was  then  represented  by  14  data  samples. 

The  averages  of  the  25  Hz  low  pass  filtered  data  and  the  60  Hz  low  pass 
filtered  data  were  then  compared.  The  filtering  operation  was  evident  for 
only  certain  stimulus  conditions  and  electrode  positions.  In  order  to 
determine  if  the  25  Hz  filtering  destroyed  information  critical  to  the  clas¬ 
sification  procedure,  a  criterion  similar  to  the  Fisher  criterion  was  com¬ 
puted.  The  criterion  was  the  ratio  of  the  average  of  the  6  between-class 
Euclidean  distances  of  the  4  mean  vectors  in  N-space  to  the  average  within- 
class  distance  of  each  sample  vector  from  the  mean  vector.  This  ratio  was 
computed  for  each  electrode  position  of  both  subjects  for  the  25  Hz  and  60 
Hz  low  pass  filtered  data  and  no  significant  differences  were  found. 

The  correlation  classifier,  as  implemented,  correlates  the  first  quar¬ 
ter  second  of  a  single  evoked  response  with  each  of  the  four  average  evoked 
responses,  adds  a  class  dependent  bias  value  equal  to  one  half  the  energy  of 
the  signal,  and  classifies  each  single  evoked  response  according  to  the 
largest  correlation  plus  bias  value  obtained. 

As  would  be  expected  from  Figures  13  and  14,  this  classifier  gave  the 
best  results  for  the  0  and  P  electrode  positions  for  both  subjects.  The 
results  of  the  correlation  classifier  using  400  single  evoked  responses  (100 
for  each  stimulus  condition)  for  the  Oz  electrode  position  are  shown  in 
Table  I. 
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One  of  the  assumptions  of  the  correlation  classifier  is  that  the  addi¬ 
tive  noise  is  Gaussian  and  white  with  the  same  spectral  density  for  all  the 
classes  involved.  This  assumption  probably  does  not  hold  very  well  for  the 
data  shown  above  because  the  data  are  time  series  data  and  the  sampling  rate 
is  over  two  times  larger  than  the  highest  frequency  present  in  the  data. 
This  means  that  adjacent  time  samples  of  the  data  and  of  the  noise  are  high¬ 
ly  correlated  and  therefore  the  noise  is  not  white. 

Table  1:  Decision  Matrix  for  the  Correlation  Classifier  Using  the  0  Elec¬ 
trode  Position.  z 

Subject  409  Subject  410 

Chosen  Class  Chosen  Class 


BF 

CF 

CT 

CB 

BF 

CF 

CT 

CB 

BF 

85 

2 

8 

5 

BF 

75 

7 

10 

8 

True 

CF 

10 

54 

8 

28 

T  rue 

CF 

17 

58 

10 

15 

Class 

CT 

22 

5 

67 

6 

Class 

CT 

12 

7 

69 

12 

CB 

16 

23 

2 

59 

CB 

12 

21 

17 

50 

The  symbols  BF,  CF,  CT,  and  CB  refer  to  the  classes  or  stimulus  condi¬ 
tions:  "Blank  Flash,"  "Checkerboard  Full  Field,"  "Checkerboard  Top  Half," 
and  "Checkerboard  Bottom  Half." 


A  prewhitening  filter  can  be  used  to  whiten  the  noise  before  it  is 
presented  to  the  correlation  classifier,  but  if  the  statistics  of  the  noise 
are  different  for  the  different  stimulus  conditions,  the  correlation  clas¬ 
sifier  may  still  not  give  a  satisfactory  performance. 

b)  Linear  Classifiers 


The  correlation  classifier  is  a  special  case  of  the  linear  classifier. 
Linear  classifiers  are  easy  to  implement  and  are  related  to  techniques  such 
as  correlations  and  Euclidean  distances.  Although  they  give  suboptimum 
results  for  many  types  of  data,  the  ease  of  implementation  and  the  simplici¬ 
ty  often  make  up  for  the  suboptimum  performance  CFukunaga,  1972D. 

The  general  linear  classifier  makes  use  of  M  vectors  V.,...,VM  and  M 
scalar  biases  b^,...,b„.  The  decision  rule  assigns  each  data  vector  X,  to 
the  class  which  has  the  Largest  discriminant  function: 

h<X)  =  vTx  +  bj  <j  =  1,...,M> 

The  objective  is  to  choose  the  coefficients  of  V.  and  the  bias  b.  such 
that  the  discriminant  function  will  have  a  large  valie  when  the  dataJvector 


X  is  a  member  of  class  j,  and  a  small  value  when  X  is  a  member  of  any  other 
class. 

c)  Quadratic  C lassi f ier 

The  Bayes  classifier  uses  a  decision  rule  based  on  a  posteriori  proba¬ 
bilities  which  is  designed  to  give  a  minimum  of  error  or  risk.  If  X  is  a 
data  vector  which  can  belong  to  any  one  of  M  classes  with  known  £  priori 
probabilities  P^/.../P„.  the  a  posteriori  probabilities  can  be  calculated 
from  the  a_  priori  probabilities  and  the  conditional  density  functions 
p(X | 1 ),..., p(x]M37  When  the  costs  of  all  the  different  types  of  errors  are 
the  same,  the  Bayes  decision  rule  to  minimize  the  error  becomes  choosing  the 
class  which  has  the  largest  a  posteriori  probability  or  discriminant  func¬ 
tion: 


h (X)  =  pCX  J  j )Pj  <j  =  1,...,M> 


Thus  the  Bayes  classifier  requires  either  a  knowledge  or  an  estimate  of 
the  conditional  probability  density  functions  p(X | 1 ),. . .,p(X | M) .  In  the 
case  where  these  density  functions  are  normal  with  expected  vectors  M-  and 
covariance  matrices  £•,  the  discriminant  functions  can  be  written  ai  qua¬ 
dratic  functions  of  X:  J 

h (X)  =  InCP  (X  |  j )P J  (j  =  1,...,M> 
hCX)  =  In  P.  -  j  In  |  -  1  (X"M.  (X-M . )  (j  =  1,...,M) 
When  XL  =  £-,  =  £,  these  quadratic  functions  reduce  to  linear  functions 

of  X: 

h(X)  =  InPj  +  Mjr'x  -  \  mJZT1  Mj  (i  = 

When  £=  I  the  identity  matrix,  the  Bayes  classifier  becomes  the  corre¬ 
lation  classifier  mentioned  earlier: 

h (X)  =  InP.  +  mTx  -  1  mTm.  (j  =  1,...,M) 

J  J  2  j  j 


Remembering  that  the  Bayes  classifier  is  optimum  in  the  sense  that  it 
minimizes  the  error,  one  can  see  that  the  linear  classifier  will  always  give 
suboptimum  performance  unless  pCX |1 ) ,. . ,,p(X |M)  are  normal  with  equal  co- 
variance  matrices.  Thus,  the  linear  classifier  is  an  optimum  classifier 
when  the  additive  noise  is  Gaussian  with  the  same  covariance  for  the  dif¬ 
ferent  classes;  and  the  correlation  classifier  is  an  optimum  classifier 
when,  in  addition,  the  additive  noise  is  white. 


In  implementing  the  maximum  likelihood  classifier  for  the  present  data 
set,  two  assumptions  are  made.  First,  all  of  the  a  priori  probabilities  P. 
are  assumed  to  be  equal.  Second,  the  conditional  density  functions  are  as^ 
sumed  to  be  normal  with  mean  vectors  approximated  by  the  average  evoked  po- 
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tential  vectors  M.|,...,M^  and  covariance  matrices  approximated  by  the  sample 
covariance  matrices  K,j,...,K^.  Although  the  assumption  of  normality  may  not 
be  entirely  valid,  the  maximum  likelihood  classifier  is  known  to  give  favor¬ 
able  results  in  many  such  practical  cases  (Glaser,  1976).  Under  the  assump¬ 
tion  of  normality,  the  discriminant  functions  which  are  the  negative  loga¬ 
rithm  of  the  a  posteriori  probabilities  can  be  written  as  (Fukunaga,  1972): 

h  j  (X)  =  (X  -  Mj  )TK  T1  (X  -  MJ  +  In  |  K  j  |  j  =  1,...,4 

which  are  seen  to  be  quadratic  functions  of  X.  The  waveform  X  is  assigned 
to  the  class  which  yields  the  minimum  discriminant  function.  Notice  that  in 
the  case  where  the  covariance  matrices  for  each  class  are  equal  to  the  iden¬ 
tity  matrix  (white  noise  for  each  stimulus  condition),  the  maximum  likeli¬ 
hood  classifier  becomes  the  correlation  classifier  described  above. 

Initially,  training  and  testing  were  done  using  all  data  points  for 
each  stimulus  condition.  Table  2  shows  the  classification  accuracy  obtained 
with  a  maximum  likelihood  classifier  using  data  from  Cz,  P  ,  0  ,  and  I 
electrodes  in  all  possible  combinations.  The  performance  is  seen  to  be  ex¬ 
ceptionally  good  for  two  or  more  electrodes  in  all  possible  combinations  and 
reasonably  good  (94%  accuracy)  for  the  single  0Z  electrode.  Similar  tests 
were  carried  out  for  the  other  test  subject  with  a  somewhat  lower  classifi¬ 
cation  accuracy  obtained. 

Classification  tests  were  carried  out  using  half  the  data  for  training 
and  half  the  data  for  testing.  Only  a  very  slight  reduction  in  accuracy  oc¬ 
curred  for  this  combination. 

Methods  of  reducing  the  number  of  features  (i.e.,  the  number  of  d?ta 
points)  used  for  classification  were  investigated.  The  technique  employed 
is  called  forward  sequential  feature  selection.  In  this  technique,  time 
samples  (or  features)  were  chosen  from  the  set  of  56  (4  electrodes  and  14 
samples  per  electrode)  by  a  forward  sequential  feature  selection  procedure. 
Six  features  were  chosen  because  they  will  provide  a  more  statistically  sig¬ 
nificant  estimation  of  the  probability  density  functions.  This  technique 
first  chooses  the  single  sample  from  among  the  56  that  gives  the  best 
discrimination  using  a  one-dimensional  maximum  likelihood  classifier. 
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Table  2:  Results  of  Maximum  Likelihood  Classifier  (Training  on  All  Data  and 
Testing  All  Features). 

Subject  409 

Electrodes  Used  %  Correct  Classification 


c 

z 

78 

pz 

79 

°z 

94 

XN 

85 

czpz 

97 

cz°z 

99 

Vn 

98 

pz°z 

99 

pzxn 

97 

°ZXH 

100 

czpz°z 

100 

Wn 

100 

CZ°ZIN 

100 

Wn 

100 

CZPZ°ZIN 

100 

The  next  sample  is  chosen  to  provide  the  best  discrimination  for  a  two- 
dimensional  classifier  using  the  previously  selected  feature  as  one  of  the 
two.  This  step  of  adding  a  selected  sample  to  the  existing  set  continues 
until  6  features  are  chosen.  This  procedure  does  not  guarantee  optimality, 
but  it  is  a  feasible  solution  to  the  problem  of  feature  selection  when  the 
original  dimensionality  is  large.  The  optimum  solution  would  be  to  try  all 
possible  combinations  of  6  features  and  would  involve  testing  32,468,436 
six-dimensional  classifiers  which  is  obviously  impractical.  The  maximum 
likelihood  classifier  was  designed  using  the  selected  samples  from  the  first 
65  potentials  in  an  experiment.  Then  60  subsequent  potentials  from  the  same 
experiment  were  classified  to  determine  the  recognition.  The  results  are 
shown  in  Table  3. 
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Table  3 


Feature 

Electrode 

Latency  (ms) 

Recognition 

1 

0 

100 

51.67% 

2 

0Z 

120 

75.00% 

3 

Pz 

120 

85.42% 

4 

Inion 

100 

87.92% 

5 

Inion 

120 

91.67% 

6 

C 

z 

100 

92.92% 

From  these  results  one  can  see  that  the  maximum  likelihood  classifier 
will  yield  a  higher  performance  using  less  data  than  the  linear  correlation 
classifier.  In  most  cases  the  linear  classifiers  have  the  advantages  of  be¬ 
ing  easier  to  implement  and  requiring  less  calculations  than  the  quadratic 
classifiers.  However  in  the  case  where  the  first  three  features  and  the  in¬ 
verse  of  the  covariance  matrix  are  known,  the  quadratic  classifier  not  only 
performs  better,  but  also  is  simplified  so  that  it  poses  no  problems  in  im¬ 
plementation.  For  this  case,  it  also  uses  less  multiplications  than  the 
linear  classifier  which  required  the  entire  data  set  to  achieve  a  comparable 
performance. 

The  features  that  were  selected  by  this  procedure  generally  correspond 
to  the  components  in  the  averages  which  show  polarity  reversals  for  some  of 
the  stimulus  conditions.  These  polarity  reversals  and  latencies  are  to  be 
expected  when  one  considers  the  results  of  the  studies  done  by  Jeffreys  et 
al  (Jeffreys  and  Axford,  1972,  I  and  II)  involving  pattern-appearance  stimu¬ 
li.  As  Donchin  et  al.  (Donchin,  1969)  have  pointed  out,  features  selected 
by  any  procedure  must  bear  some  connection  to  the  underlying  process  before 
any  relationship  between  the  discriminant  functions  and  the  evoked  poten¬ 
tials  for  the  different  stimulus  conditions  is  to  be  inferred. 

The  forward  sequential  feature  selection  procedure  was  then  performed 

on  the  combinations  of  three  class  data  obtained  by  eliminating  one  class. 

These  results  are  shown  in  Table  4.  The  first  few  features  selected  in 

Table  4  are  again  at  100  and  120  ms  in  electrodes  0  and  P  .  But  now  other 

z  z 

latencies  are  evident  especially  when  the  CF  stimulus  condition  is  eliminat¬ 
ed. 


Table  4 


Class 

Eliminated:  BF 

Feature 

Electrode 

Latency  (ms) 

Recognition 

1 

0 

100 

62.22% 

2 

°z 

120 

81.11X 

3 

Inion 

100 

81.11% 

4 

P 

220 

88.89% 

5 

Inion 

140 

89.44% 

6 

Inion 

140 

90.00% 

Class 

Eliminated:  CF 

Feature 

Electrode 

Latency  (ms) 

Recognition 

1 

P 

100 

62.22% 

2 

0Z 

100 

76.67% 

3 

0Z 

140 

88.33% 

4 

Cz 

140 

92.78X 

5 

cz 

180 

95.56% 

6 

Inion 

180 

96.67% 

Table 

4  (Continued) 

Class 

Eliminated:  CT 

Feature 

Electrode 

Latency  (ms) 

Recognition 

1 

0 

100 

63.33% 

2 

0Z 

120 

77.22% 

3 

Pz 

120 

86.11% 

4 

Inion 

100 

88.89% 

5 

Inion 

120 

93.33% 

6 

C 

z 

100 

93.89% 

Class 

Eliminated:  CB 

Feature 

Electrode 

Latency  (ms) 

Recognition 

1 

0 

100 

66.67% 

2 

0Z 

120 

85.00% 

3 

Pz 

120 

92.22% 

4 

Pz 

160 

95.00% 

5 

Cz 

200 

95.00% 

6 

cz 

z 

160 

95.56% 

Since  the  averages  shown  in  Figure  13  are  the  summation  of  many  evoked 
potentials,  one  cannot  ascertain  much  about  the  components  in  a  single  po¬ 
tential  from  the  average  alone.  In  order  to  obtain  more  information  bout 
the  single  evoked  potentials,  these  components  were  located  by  a  procedure 
outlined  by  McGillem  and  Aunon  (McGillem  and  Aunon,  1977),  and  a  histogram 
of  the  number  of  peaks  detected  at  each  latency  was  constructed.  Figure  15 
shows  the  matrix  of  histograms  for  each  electrode  and  stimulus  condition. 
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In  many  cases  the  general  shape  of  the  histogram  bears  some  relation  to  the 
average  evoked  potential.  Notice  that  the  histograms  for  the  C  electrode 
and  CF  or  CB  stimulus  condition  are  generally  flat  in  sfiape.  The 
corresponding  averages  shown  in  Figure  13  also  tend  to  have  less  sharply  de¬ 
fined  peaks.  From  Table  4  one  can  see  that  when  the  CF  or  CB  stimulus  con¬ 
dition  is  eliminated,  two  of  the  features  selected  are  from  C  compared  with 
one  or  none  in  the  other  two  cases.  2 

The  components  with  polarity  reversals  evident  in  Figure  3  are  also 
present  in  Figure  14.  Thus  the  peaks  which  show  polarity  reversals  between 
the  stimulus  conditions  are  present  in  the  single  potentials  as  well  as  in 
the  averages.  These  peaks  correspond  to  the  features  chosen  by  forward 
sequential  feature  selection  correspond  to  components  having  the  largest 
frequency  of  occurrence  in  the  single  potentials. 

Note  that  the  recognition  values  in  Table  3  correspond  not  only  to  the 
single  feature  given  but  also  to  all  the  features  preceding  it.  So  two 
selected  features  from  100  and  120  ms  in  0  gave  discrimination  comparable 
to  the  entire  set  of  time  points  in  0  .  this  is  understandable  from  Figure 
2  in  that  many  latencies  show  an  approximately  equal  number  of  potentials 
having  both  positive  and  negative  peaks.  The  amplitudes  at  these  latencies 
will  present  little,  if  any,  discriminant  information  to  the  classifiers. 

Task  4,3^ 

"Subsequent  to  the  preliminary  investigations,  data  acquisition  for  the 
latter  phases  of  this  contract  is  to  be  completed  at  the  Visual  Display  Sys¬ 
tems  Branch  of  AMRL.  This  will  require  that  a  controlled  interface  be  esta¬ 
blished  between  the  recording  systems  at  the  contractor's  laboratory  and 
those  at  AMRL.  Consequently,  at  least  two  trips  by  the  contractor  to  AMRL 
(two  investigators)  and  two  trips  by  AMRL  personnel  to  the  contractor’s  la¬ 
boratory  (two  investigators)  will  be  necessary  to  ensure  complete  compati¬ 
bility  of  hardware.  Specifications  will  be  agreed  upon  for:  (a)  type  of  di¬ 
gitization  procedure;  (b)  data  encoding;  (c)  data  calibration;  and  (d)  EEG 
amplification  settings." 

This  task  was  completed  as  the  data  were  transferred  between  AMRL  and 
the  contractor's  laboratory.  Specifications  concerning  data  recording,  cod¬ 
ing,  calibration  and  digitization  were  completed. 

Tasks  4.4,  4.5^  and  4.6^ 

4.4  "The  initial  tests  to  be  performed  at  AMRL  will  utilize  the 
stimulus  events  and  electrode  placements  determined  in  sections  4.1  and  4.2 
above.  Digital  tapes  will  be  submitted  to  the  contractor  and  signal  pro¬ 
cessing  studies  will  commence  at  this  point.  Data  processing  techniques 
developed  in  section  4.2  above  will  be  improved  with  the  highly  controlled 
evoked  potential  activity  recorded  at  AMRL.  Other  single  event  evoked  po¬ 
tential  procedures  are  to  be  considered,  such  as  the  use  of  a  latency 
corrected  average  which  takes  into  account  time  related  fluctuations  of  in¬ 
dividual  components  of  the  single  response.  Further,  the  use  of  selected 
segments  of  the  latency  corrected  average  (100-250  msec  after  stimulus  on¬ 
set)  as  a  matched  filter  for  the  detection  of  corresponding  components  in 
the  single  response  should  be  investigated.  Following  the  completion  of 
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these  tasks,  the  single  responses  associated  with  the  different  chromatic 
stimulus  events  will  be  accurately  characterized." 

4.5  "The  “features"  of  the  single  responses  elucidated  in  section  4.4 
above  will  form  the  basis  for  classification  according  to  stimulus  type. 
Some  of  the  techniques  to  be  utilized  in  the  study  of  this  relationship  are 
the  linear  least  squares  multidimensional  classifier  and  the  maximum  likeli¬ 
hood  classifier.  These  techniques  and  the  software  developed  for  their  im¬ 
plementation  will  be  tested  in  the  remaining  subjects  of  this  initial  test¬ 
ing  session.  The  possibilities  of  utilizing  a  filter  "common"  to  all  sub¬ 
jects  will  also  be  explored.  The  existence  of  such  a  filter  would  imply 
that  no  £  priori  information  concerning  a  specific  subject  would  be  needed 
in  order  to  specify  (within  some  statistical  bound)  the  color  of  the 
stimulus.  This  would  be  advantageous  because  then  no  initial  testing  for  a 
specific  subject  would  be  required  prior  to  the  actual  experiment." 

4.6  "A  new  set  of  experiments  will  be  conducted  at  AMRL  utilizing  sub¬ 
jects  not  used  in  the  first  set  of  experiments.  The  theories  and  hypotheses 
developed  earlier  will  be  tested  with  this  new  set  of  data  and  changes  or 
alterations  will  be  made  at  this  time  in  the  software  to  optimize  the  per¬ 
formance  of  the  data  processing  algorithms.  The  digital  tapes  submitted  by 
AMRL  to  the  contractor  (in  section  4.4  above)  will  remain  at  that  facility 
until  the  completion  of  this  contract.  The  attached  program  schedule  indi¬ 
cates  the  phasing  of  tasks  to  be  completed  under  this  contract." 

A  new  set  of  tests  were  performed  at  AiiPL  in  order  to  verify  the  use¬ 
fulness  of  the  previously  developed  classification  procedures  to  different 
and  more  complex  situations.  First,  the  tests  performed  earlier  were  ex¬ 
panded  to  include  quadrants  and  right  and  left  visual  fields.  The  number  of 
electrodes  used  was  also  increased  to  incorporate  electrodes  off  the  mid¬ 
line.  Electrodes  were  positioned  in  the  right  and  left  parietal  cortex  and 
the  right  and  left  occipital  cortex.  Stimulus  parameters  were  as  follows: 

(1)  The  entire  CRT  screen  abruptly  becomes  a  checkerboard  of  checksize  10' 
for  a  duration  of  0.5  seconds.  The  screen  then  returns  to  the  quies¬ 
cent  state  for  1.5  seconds.  Space  average  luminance  remains  constant 
throughout  the  2  seconds. 

(2)  Same  as  (1)  except  left  half  of  screen  becomes  a  checkerboard,  right 
half  stays  blank  (quiescent). 

(3)  Same  as  (1)  except  right  half  of  screen  becomes  a  checkerboard,  left 
half  stays  blank  (quiescent). 

(4)  Same  as  (1)  except  upper  half  of  screen  becomes  a  checkerboard,  lower 
half  stays  blank  (quiescent). 

(5)  Same  as  (1)  except  lower  half  of  screen  becomes  a  checkerboard,  upper 
half  stays  blank  (quiescent). 

(6)  Same  as  (1)  except  upper  left  quadrant  of  screen  becomes  a  checker¬ 
board,  the  other  quadrants  stay  blank  (quiescent). 


a  checker- 


17)  Same  as  (1)  except  upper  right  quadrant  of  screen  becomes 
board,  the  other  quadrants  stay  blank  (quiescent). 

(8)  Same  as  (1)  except  lower  left  quadrant  of  screen  becomes  a  checker¬ 
board,  the  other  quadrants  stay  blank  (quiescent). 

(9)  Same  as  (1)  except  lower  right  quadrant  of  screen  becomes  a  checker¬ 
board,  the  other  quadrants  stay  blank  (quiescent). 

A  minimum  of  200  repetitions  of  each  stimulus  were  presented  to  each  of  5 
subjects.  Time  between  the  onset  of  stimulus  repetitions  equaled  2  seconds. 

The  above  tests  are  expected  to  provide  a  more  detailed  experimental 
evaluation  of  the  techniques  developed  earLier  by  increasing  to  nine  the 
choices  available  to  the  classifier.  The  electrodes  positioned  off  the 
mid-line  are  expected  to  provide  the  extra  features  needed  for  accurate 
classification. 

In  addition  to  the  above  tests,  the  following  tests  involving  underly¬ 
ing  psychological  factors  were  also  performed. 

(10)  V  Inverted  V 

In  this  test,  the  subject  views  the  alphabet  letter  V  and  occasionally 
the  same  letter  inverted.  Interstimulus  intervals  were  uniformly  distribut¬ 
ed  between  2  and  5  seconds  in  order  to  negate  build  up  of  the  contingent 

negative  variation  (CNV).  Probability  of  occurrence  for  the  inverted  letter 

was  20%  and  for  the  normal  letter  80%.  "On-time"  for  each  letter  was  0.5 
seconds  and  recording  of  the  EEG  was  coincident  with  the  presentation  of 
each  Letter. 

(11)  Sternberg  Paradigm  Test 

During  this  test,  the  subject  was  given  two  letters  to  memorize  (such 

as  T  and  F) .  These  were  called  relevant  letters  (RL),  the  rest  oi  the  al¬ 

phabet  comprising  the  nonrelevant  letters  (NRL) .  The  subject  was  then  shown 
a  letter  and  asked  to  make  a  decision  as  to  whether  it  was  an  RL  or  an  NRL, 
the  decision  involving  pushing  one  of  two  provided  buttons.  The  inter¬ 
stimulus  interval  was  approximately  5  seconds.  The  RL  had  a  50%  probability 
of  occurrence  and  NRL  a  50%  chance  of  occurring.  Recording  of  the  EEG  was 
coincident  with  the  presentation  of  each  letter. 

(12)  Focused  and  Defocused  Letters 

During  this  test,  the  subject  was  shown  five  letters  of  the  alphabet, 
one  at  a  time,  200  times  each.  The  letters  (I,  R,  B,  0,  C)  were  shown  first 
as  sharply  defined  and  then  as  defocused  letters.  Recording  of  the  EEG  was 
coincident  with  the  presentation  of  each  letter. 

(13)  Edge  Watching 

During  this  test,  the  subject  was  shown,  side  by  side,  two  slides  of  a 
photographic  image.  One  of  the  slides,  always  shown  on  the  left,  was  known 
to  be  ground-truth  level.  The  subject  was  then  asked  to  make  a  subjective 
judgment  as  to  whether  the  test  slide  was  more  or  less  sharply  defined  than 


the  ground-truth  slide.  Once  this  decision  process  terminated,  both  sides 
were  flashed,  the  EE6  recorded  at  this  time,  and  the  subject  asked  whether 
these  2  slides  were  the  same  ones  he  had  previously  observed. 

RESULTS 

Checkerboard  Stimulation 

Conditions  1  through  9  were  digitized  and  filtered  following  the  pro¬ 
cedure  outlined  earlier.  The  results  of  applying  the  quadratic  classifier 
to  the  data  are  summarized  in  the  Appendix.  These  results  were  not  as 
dramatic  as  those  reported  earlier,  but  no  further  data  processing  was  per¬ 
formed  on  the  data  in  order  to  allow  time  to  analyze  data  from  the  other  mo¬ 
dalities  of  stimulation.  Out  of  the  four  conditions  left,  namely  10  through 
13,  most  of  the  effort  was  concentrated  on  the  Edge  Matching  experiment. 

Due  to  poor  data  (V  inverted  V)  and  lack  of  time  (Sternberg  test),  it 
was  not  possible  to  analyze  the  data  from  experiments  10  and  11. 

Focused  and  Defocused  Letters  Classification  Results 

The  tests  performed  attempted  to  classify  blurred  against  focused 
letters  presented  to  the  subject.  A  quadratic  classifier  was  used  as  the 
classification  algorithm  and  the  results  achieved  are  shown  in  Table  5. 

Table  5 

T raining  and  Classification  on  Same  Samples 
Class  1:  R  focused  Class  2:  R  blurred 

Feature  Electrode  -  Latency  X  Correctly  Classified 


1 

°z 

-  200 

MS 

66 

2 

IN 

-  500 

MS 

70 

3 

°z 

-  300 

MS 

74 

4 

Cz 

-  200 

MS 

76 

5 

Cz 

-  160 

MS 

77 

6 

o. 

-  380 

MS 

78 

-  38  - 


i 


Class  1: _ B  focused 


Feature 

1 

2 

3 

4 

5 

6 


Electrode  ~  Latency 

C  -  240  MS 
z 

0  -  500  MS 

z 

C  -  300  MS 
z 

C  =  220  MS 
z 

C  -  180  MS 
z 

INION  -  540  MS 


Class  2:  B  blurred 
X  Correctly  Classified 
65 
72 
74 

76 

77 
77 


Class  1:  0  Focused 


Feature 

1 

2 

3 

4 

5 

6 


Electrode  -  Latency 

0  -  260  MS 

z 

INI0N  -  340  MS 
INION  -  140  MS 

0  -  80  MS 

z 

0  -  120  MS 

z 

0  -  0  MS 

z 


Class  2:  0  Blurred 
%  Correctly  Classified 
61 
62 
67 

70 

71 
70 


Class  1:  R  focused 
Feature  Electrode  -  Latency 


1 

2 

3 

4 

5 


P 

z 

0 

z 

C 

z 

0 

z 

INI0N 

P 

z 


-  260  MS 

-  480  MS 

-  220  MS 

-  600  MS 

-  40  MS 


Class  2;  0  focused 

X  Correctly  Classified 
59 

65 

66 
68 
69 


«"v 

w- 


6 


-  360  MS 


70 


The  first  three  tests  attempted  to  separate  the  same  focused  and 
blurred  letters.  The  last  test  attempted  to  separate  two  letters  that  were 
focused.  The  classification  accuracy  obtained  in  these  tests  was  approxi¬ 
mately  70  percent.  This  indicates  that  some  type  of  data  preprocessing 
similar  to  the  one  used  to  be  described  in  the  edge-matching  experiments 
should  be  investigated. 


Edge  Matching  Classification  Results 


There  were  five  possible  gradations  of  sharpness  numbered  1,  7,  11,  17 
and  22.  The  ground-truth  slide,  which  always  appeared  on  the  left, 
corresponded  to  number  eleven.  Number  1  slide  was  the  sharpest  and  22  the 
least  focused.  Therefore,  the  subject  was  shown  slide  11  on  the  left  and 
one  of  the  other  five  on  the  right.  Throughout  the  test,  the  average  lumi¬ 
nance  level  observed  by  the  subject  was  kept  constant  to  minimize  any  inten¬ 
sity  changes  in  the  evoked  potential  caused  by  defocusing  of  a  slide.  Using 
the  quadratic  classifier,  it  was  then  attempted  to  discriminate  among  the 
test  images  using  the  evoked  potentials.  A  class  was  made  up  of  combina¬ 
tions  of  slides  such  as:  Class  1  made  up  by  showing  slides  11  and  11  and 
Class  2  from  slides  11  and  22,  etc...  The  first  approach  was  to  classify 
the  individual  evoked  potentials  produced  by  the  two  extreme  slides.  These 
two  slides  corresponded  to  the  sharpest  and  the  least  sharply  defined  images 
(slide  11  was  always  shown  on  the  left),  i.e.  Class  1  =  11/1,  Class  2  = 
11/22.  The  results  are  shown  on  Table  6. 

Table  6 

T raining  and  classification  on  same  samples 
Feature 


1 

C 

(300 

msec) 

71.4 

2 

Pz 

(280 

msec) 

76.4 

3 

IZ 

(100 

msec) 

79.3 

4 

cn 

(240 

msec) 

82.1 

5 

cz 

(  60 

msec) 

84.3 

6 

(360 

msec) 

85.7 

7 

(500 

msec) 

89.3 

8 

pz 

(100 

msec) 

90.0 

9 

S 

(460 

msec) 

91.4 

10 

(320 

msec) 

92.7 

As  it  may  be  observed  with  8  features,  the  percent  correct  classifica¬ 
tion  is  on  the  order  of  90%.  It  is  also  interesting  to  observe  that  if  only 
the  first  feature  is  considered,  i.e.,  C  3  300  msec,  the  classification  ac¬ 
curacy  is  71%.  In  this  procedure  training  and  testing  were  carried  out  us¬ 
ing  all  samples. 

Training  on  half  of  the  samples  -  Testing  on  half  of  the  samples. 
Class  1  -  11/1,  Class  2  -  11/22  when  only  the  first  half  of  the  data  set  is 
used  for  training  the  classifier  and  then  testing  is  carried  out  over  the 
second  half  of  the  data  set  there  is  a  reduction  in  performance  as  shown  in 
Table  7. 


T$sr 


-  AO  - 


Training  and  classi 
Feature 

1  C  (300  msec) 

2  1 2  (100  msec) 

3  Pn  (120  msec) 

A  02  (160  msec) 

5  O2  (2A0  msec) 

z 


Table  7 

:ication  on  different  samples 

X  Correctly  Classified 

67.1 

75.7 
82.9 

85.7 

87.1 


The  next  test  attempted  to  classify  between  the  sharpest  slide  (1)  and 
the  standard  slide  (11),  i.e.  Class  1  =  11/11  and  Class  2  =  11/1.  The 
results  are  shown  on  Table  8. 


Table  9 

T raining  and  classification  on  different  samples 


Feature 


1 

C 

(280 

msec) 

62.9 

2 

c2 

(  0 

msec) 

65.7 

3 

c2 

(200 

msec) 

70.0 

A 

o2 

(580 

msec) 

72.9 

5 

o2 

(  83 

msec) 

78.6 

6 

c2 

(3A0 

msec) 

78.6 

7 

I2 

(120 

msec) 

81.  A 

8 

on 

(  20 

msec) 

81.  A 

9 

p2 

(3A0 

msec) 

82.8 

10 

p2 

(5  A0 

msec) 

8A.3 

It  is  difficult  to  interpret  the  second  feature  selected  by  the  clas¬ 
sifier,  except  that  this  is  one  out  of  three  possible  choices  (alternate 
trees)  at  this  level.  It  apparently  represents  a  stable  point  in  the 
response  by  which  changes  at  other  points  can  be  measured. 

The  next  test  attempted  to  classify  between  the  standard  and  the  least 
sharp  slides.  Class  1  =  11/11  and  Class  2  =  11/22.  The  results  are  shown  on 
Table  9. 


Training  and  classi 
Feature 

1  C  (280  ms) 

2  I2  (  80  ms) 

3  0n  (200  ms) 

A  P2  (  A0  ms) 

5  C2  (  60  ms) 

z 


Table  9 

ication  on  different  samples 

X  Correctly  Classified 
71 
77 
81 
8A 
83 


Although  the  results  were  adequate,  they  did  not  have  the  degree  of 
correct  classification  that  was  desired,  so  several  techniques  of  data  pro¬ 
cessing  were  explored. 
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Acceptance  or  Rejection  of  Single  Responses  According  to  Thei r  Euclidean 
Distance  from  the  Ensemble  Mean 

After  the  mean  and  standard  deviation  of  the  Euclidean  distances 
between  each  single  response  and  the  ensemble  mean  were  calculated,  those 
single  responses  whose  distances  exceeded  1  standard  deviation  were  reject¬ 
ed.  Results  of  the  classification  on  the  remaining  responses  (10%  of  the 
responses  were  rejected)  are  shown  in  Table  10. 


1 

2 

3 

4 

5 


Table  10 

T raining  and  classification  on  different  samples 
(Class  1  =  11/1,  Class  2  =  11/22) 

Feature  %  Correctly  Classified 

0  (440  ms)  70 

PZ  (120  ms)  75 

CZ  (300  ms)  78 

Iz  (100  ms)  83 

0  (  40  ms)  85 


The  increase  in  correctly  classified  responses  was  not  very  signifi¬ 
cant  . 


Improved  MMSE  Filtering 


Procedure 


By  employing  a  more  powerful  filtering  technique  designed  to  separate 
the  evoked  potential  from  the  ongoing  EE6  it  should  be  possible  to  obtain 
performance  comparable  to  that  which  would  be  obtained  for  higher  SNR.  One 
filter  that  has  been  designed  for  this  purpose  is  the  minimum  mean  square 
error  filter  (MMSE)  which  makes  use  of  the  autocorrelation  matrix  of  the 
data  and  an  estimate  of  the  autocorrelation  function  of  the  signal  waveform 
CMcGillem,  1977D.  In  previous  experiments  with  the  MMSE  filter  the  auto¬ 
correlation  function  of  the  signal  waveform  was  approximated  by  the  auto¬ 
correlation  function  of  the  average  evoked  potential.  This  led  to  the 
design  of  useful  MMSE  filters  but  not  too  high  performance  filters  because 
much  of  the  detail  of  the  signal  waveform  is  lost  in  the  averaging  process. 


As  a  means  of  improving  the  performance  of  the  MMSE  filter  a  new  pro¬ 
cedure  for  estimating  the  autocorrelation  function  of  the  evoked  potential 
waveform  was  developed.  This  method  made  use  of  the  latency  corrected  aver¬ 
age  (LCA)  which  is  obtained  in  the  following  manner.  Each  individual 
response  in  an  ensemble  of  MMSE  filtered  evoked  potentials  is  searched  by 
computer  to  locate  the  individual  peaks.  The  peaks  are  then  clustered  by  a 
statistical  procedure  into  groups  corresponding  to  the  same  portion  of  the 
response  waveform.  Within  each  group  the  mean  latency  is  determined  and  the 
corresponding  segments  of  the  original  waveforms  are  aligned  to  that  latency 
and  averaged  giving  the  LCA  for  that  component.  This  is  repeated  for  each 
cluster  giving  the  complete  LCA.  When  computed  in  this  manner  the  LCA  is  a 
set  of  disjoint  waveform  segments  corresponding  to  the  components  in  the 
evoked  potential.  Figure  16  shows  a  superposition  of  the  conventional  aver¬ 
ages  and  the  LCAs  for  four  electrode  positions  for  the  image  comparison 
evoked  potentials.  Prior  to  processing,  the  data  shown  were  filtered  with  a 
30  Hz  low  pass  filter. 


Figure  16. 


Superposition  of  the  Convent 
tered  C30  Hz]  Data  and  their 


In  order  to  be  useful  in  the  design  of  an  improved  MMSE  filter,  it  is 
necessary  to  convert  the  disjoint  LCA  into  a  continuous  waveform  so  that  its 
autocorrelation  function  can  be  computed.  Several  methods  of  accomplishing 
this  were  tried,  and  it  was  found  that  excellent  fits  in  the  immediate  vi¬ 
cinity  of  the  LCA  segments  could  be  obtained.  However,  when  one  or  more 
components  were  missing  the  approximating  function  behaved  erratically. 
This  is  illustrated  in  Figure  17  where  the  LCA  of  Figure  16  was  fitted  with 
a  sum  of  sinusoids.  The  large  peaks  in  Figures  17A  and  17B  occur  where 
there  are  no  components  of  the  LCA  present  and  are  obviously  not  a  valid  ap¬ 
proximation  in  that  region.  Also,  Figure  17D  looks  unnatural  because  of  the 
presence  of  only  low  frequency  components.  This  latter  effect  was  the 
result  of  there  being  only  a  limited  number  of  components  that  can  be  unam¬ 
biguously  computed  due  to  the  relatively  few  components  present  in  the  LCA. 
In  the  weighted  least  squares  fitting  procedure  used  the  frequencies  of  the 
components  are  selected  first  (the  number  of  different  frequencies  possible 
is  determined  by  the  amount  of  data  available)  and  then  the  amplitude  deter¬ 
mined.  By  eliminating  those  components  that  have  very  small  coefficients 
and  adding  new  components  not  previously  used,  an  improved  fit  can  be  ob¬ 
tained  by  several  iterations.  At  the  same  time  by  setting  the  values  of  the 
data  in  the  vicinity  of  missing  components  to  a  small  value  (typically 
zero),  but  using  a  greatly  reduced  weighting  factor  (1/10)  in  this  region 
relative  to  the  rest  of  the  data  set,  the  generation  of  spurious  peaks  can 
be  eliminated.  The  results  of  four  iterations  of  the  fitting  procedure  gave 
the  continuous  LCA  waveforms  shown  in  Figure  18.  Comparison  of  the  continu¬ 
ous  LCA  waveforms  with  the  disjoint  LCA  waveforms  of  Figure  16  shows  excel¬ 
lent  agreement. 

As  an  illustration  of  the  increased  filtering  action  that  resulted  from 
redesigning  the  MMSE  filter  using  the  autocorrelation  function,  of  the  con¬ 
tinuous  LCA  to  represent  che  signal  autocorrelation  function  the  first  indi¬ 
vidual  response  for  the  four  electrodes  are  shown  in  their  original  (30  Hz 
low  pass  filtered)  form  in  Figure  19  and  as  filtered  by  the  improved  MMSE 
filter  in  Figure  20.  It  is  seen  that  the  overall  trends  are  retained  with  a 
substantial  enhancement  of  the  individual  components.  It  is  this  signal 
enhancement  that  is  expected  to  lead  to  improved  classification  performance. 

Classification  of  Improved  MMSE  Filtered  Data 

In  order  to  investigate  the  effect  of  improved  filtering  on  classifica¬ 
tion  accuracy  the  previous  tests  relating  to  image  comparison  were  repeated 
after  filtering  the  data  with  the  improved  MMSE  filter.  The  classification 
procedure  was  as  follows.  The  MMSE  filter  was  computed  for  data  correspond¬ 
ing  to  one  of  the  classes  (Class  1:  sharpest  image)  and  used  to  filter  the 
data  of  both  Class  1  and  Class  2  (least  sharp  image).  The  first  half  of  the 
data  was  then  used  to  train  the  quadratic  classifier  and  the  second  half 
used  to  determine  the  classification  accuracy.  The  results  are  shown  in 
Table  .  Comparing  these  results  with  those  given  in  Table  11  shows  that 
the  error  rate  has  been  reduced  by  more  than  50X.  It  seems  likely  that  by 
refining  this  procedure  even  further  improvements  in  performance  should  be 
possible. 


Table  11 

Classification  Performance  on  the  Image  Comparison  Test  After 
Processing  the  Data  with  the  Improved  MUSE  Fi Iter 
(Class  1:  Sharpest  Image  -  Class  2:  Least  Sharp  Image) 


Feature  Electrode  -  Latency  %  Correctly  Classified 


1 

0 

z 

-  160 

MS 

76 

2 

0 

z 

-  0 

MS 

81 

3 

P 

z 

-  0 

MS 

90 

4 

Inion 

-  560 

MS 

91 

5 

C 

z 

-  140 

MS 

94 

CONCLUSIONS 

The  purpose  of  this  research  program  was  to  demonstrate  the  feasibility 
of  on-line  classification  of  evoked  cortical  potentials  associated  with 
specific  stimulus  events.  For  the  initial  experimental  phase  a  number  of 
stimuli  (checkerboard  patterns)  were  selected  which  showed  pattern  reversals 
and  other  noticeable  feature  variations  in  their  conventional  average  evoked 
potentials.  The  assumption  was  made  that  the  dissimilarities  of  the 
features  on  the  average  would  carry  to  the  single  potentials. 

Several  classification  techniques  were  investigated  and  tested  using 
the  single  evoked  potentials.  A  Bayes  classifier  utilizing  a  quadratic 
discriminant  function  gave  the  best  results.  Classification  accuracies  were 
on  the  order  of  66%  for  a  linear  classifier  as  compared  to  94%  for  a  qua¬ 
dratic  classifier  when  the  optimum  number  of  features  were  used. 

Following  these  tests,  the  experimental  measurements  were  extended  in 
two  directions.  First,  a  more  elaborate  battery  of  tests  involving  checker¬ 
board  patterns  was  generated.  These  tests  involved  right  and  left  visual 
fields,  upper  and  lower  visual  fields  and  quadrants  of  each  field.  The 
results  of  these  tests  are  included  in  the  Appendix  of  this  Report. 

Second,  a  series  of  new  experiments  involving  more  complex  experimental 
paradigms  was  generated.  The  aim  now  was  to  test  the  previously  developed 
programs  and  algorithms  and  to  investigate  other  techniques  such  as  data 
preprocessing.  Two  of  the  new  paradigms  that  were  investigated  were  focus- 
defocused  letters  and  edge  matching. 

Results  obtained  in  the  edge  matching  experiment,  fully  described  in 
the  preceding  sections,  were  very  encouraging  giving  classification  accura¬ 
cies  above  80%.  By  using  a  newly  developed  data  preprocessing  technique  the 
error  rate  was  further  reduced  by  more  than  50%.  This  type  of  data  prepro¬ 
cessing  should  be  investigated  further  to  assess  its  usefulness  in  other 
paradigms  such  as  in  the  focus-defocused  letters  experiment. 
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APPENDIX 


The  following  are  the  classification  results  obtained  for  conditions  1 
through  9  as  outlined  in  the  Results  section  of  this  Report.  All  of  the 
results  reported  below  were  obtained  by  training  and  classifying  on  dif- 


ferent  samples. 

Subject  425 

Subject  —  90  training/90  testing 
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